Less is more: optimal learning with subsampling regularization∗

In this talk, we discuss recent results on common techniques for scaling up nonparametric methods such as kernel methods and Gaussian processes. In particular, we focus on data dependent and independent sub-sampling methods, namely Nystrom and random features, and study their generalization properties within a statistical learning theory framework. On the one hand we show that these methods can achieve optimal learning errors while being computational efficient. On the other hand, we show that subsampling can be seen as a form of regularization, rather than only a way to speed up computations. [Joint work with Raffaello Camoriano, Alessandro Rudi.]